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1 Introduction 

Oadie manager (CM) caches and maintains dynamic file (template) output in a disk cache for use by a 
Web Server (WS) or a Page Generator (PG). Page generator is a generic term for Ihe engine that 
executes a template (servlet engine, HS instance, Tel engine). The output is received ftom the PG filter 
in placement messages.via Hypertext Transfer Protocol (HTTP). 

The cached output of a template is' called a cache file or a template instance. 

stage ^ content delivery server-^^^ — cache 

1 . . ■ . 1 - • 

• 1+ ■ ■ 1 ■ • 

deployment group cache manager ' 

+ ■ ■ . ■ ■ • 



+ ■■'.' ' . . 

site - — ■■ web application 

'. . • 1 "+ ■ . . . 

A stage is associated with 1 or more deployment groups (DG). A DG is associated with only 1 stage. 

A stage contains 1 or more Coritisnt DeUvery Servers (CDSs). A CDS is.in only 1 stage. 

A. site is a set of related content and web applications that progresses through stages of development.' A 
stage is associated with 1 or more sites eifher through multiple DGs per stage or multiple sites per DG. 
Multiple DGs per stage are required when sites, within a stage must be treated differently. An example 
is WS virtual hosting where each site's virtual domain has a separate docroot that receives only that ' 
site's content' A site is associated with 1 or more DGs. with eiach DG in a different stage. 

A site has 1 or more web applications. A web application generates a single site. 

A CM manages 1 cache. A cache can either be a WS cache, a PG cache, or both. A cache conteins. 
content from 1 or more stage, CDSs.' A CDS's content can' be cached in 1 or more caches. 

A template is designated as cacheable by adding an entry to the Cache Definition File (CDF) that is 
deployed to the Application Manager (AM). There is 1 CDF per web app. AM sends per-template 
definition information to CM in metadata messages via the Persistent HTTP Messaging J¥amewOTk " 

..... 

CM saves the tanplate-levei cache definition information in the metadata database. As each template' 
instance is cached, CM saves the corresponding instance-level metadata as well. Portions of the 
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template-level and instance-level metadata are replicated for read-only use by the WS plug-in and PG 
filter/. 

Expiration of a cache file means it is old. Depending on the regeneration policy, the file may be deleted, 
or the template may be re-executed to produce a new version of the cache file. Events that cause 
expiration are template update, periodic expiration, content change, or the clear cache API command 

Regen^ation is the automatic re-execution of a template to produce a new version of a cache file. 
Regeneration is triggered by expiration. The regeneration request will contain the cache-sensitive 
request parameters saved fiom ^e request that originally caused the instance to be cached. Other 
parametra like the Content-type header are also included in the regen^ation request 

Both expiration and regeneration are controlled by Policy Definition File (PDF) entries. There is 1 PDF 
file per site. AM sends per-template policy information to CM in metadata messages. 

LI Instance State diagram 

CM clients will only serve from the cache when the state is Valid or Reg^. • 
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2 Concurrency architecture 

The following diagram shows the threads and thread pools that process the basic CM functioDS. 
Instance deleter and pmodic expiration are single time-driven threads. Placement, template changes, 
record changes, and dear cache messages are all handled by the HttpAction framework thrpad pool 
whose size is determined by the POOL^SIZE configuration variable. Regeneration requests, responses, 
and placements are handled by a separate thread pool whose size is determined by the 
REGENBRATE.CONCURRENCYJUtt^ 

Periodic expiration, placement, template changes, record changes, and clear cache can all send 
regenerate messages to the regenerator pool. 



HTTP messages 




11 



Vignette Corporation Confidential 



Q 



3 Protocols 

5./ AM-CMmetadata-msg 

Tlie XML schema of the body of die HTTP metadata message is in 

..V AcomVvi gnette\cod\cm\metadataMsg.xsd . The XML docmnent describes the addition of templates, . 
the deletion of templates, metadata changes, tenq)late changes, and simultaneous changes to a template 
and its metadata. MetadataMsgHandler on page 13 shows how the messages are handled. 

32 AM-CM record-update-msg 

TTie XML schema of the body of the HTTP record update message is in 

.\;\com\vignette\cod\cm\tecordUpdateMsg.xsd . RecordUpdateMsgHandler on page 19 shows how the 
messages are handled. ' 

33 PG-CM placement-msg 

The XML schema of the XML portion of the placement-msg is in 

..V AcomXvi gnette\cod\cm\pIacementMsg.xsd , The XML document described by the schema is 
embedded in the 1** part of an HTTP 1-1 multipart/mixed message whose subsequent parts contain the 
template outputs to be cached. Each p^ has aiCqiitent-'Iengfh he^ider that specifies the # of bytes in the 
part. The XML document in the 1^ part describes the t^plate^outppts in the other parts of the HTTP 
, message and their ijijeirelationships. PlacementMsgHandletPn page 21 shows how the messages are.' 
handled. . 

3:4 PG-CM clear''Cach€''mSg '* / 

.The XML schema of the body of the clear cache message is in 

. .V AcQm\vi gneite\cod\cm\clearCacheMs g.xsd . The XML document described by the schema is 
embedded in eithef the body of an HTTP message. CleaiCacheMsgHandler on page 32 shows how the 
messages are handled. . • 



G 
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4 Message handlers 

4.1 MetadataMsgHandler 

Handles the AM-CM metadaCa-msg described on page 12. 

The following diagram shows the classes ihat process the metadata-msg: 





MertadataMoQHBndler 



•i:proce8sM6dataMsg(htipBody:lnputStrBain, oontsntLengthTlnf) 
•atkiTemplate(alaoeld:Strina, cdsldrString, altekl:8Wno, dald:String,tEniry:T9nip»ateEntry) 
-dB)stoTemplate(8ta3eId:8trtr)g. cdsldkSting. 8Hold:8lring, tEntoytTamplBteEntry) 
'IiwandatdAiIlnstanc8s(tl»Mrre7nplat8M8iadbta)' 
'lnvaS(lBlBEachIn8tance(ln8tances:Ll$Q 

■updateTBmptoMotadataCstaDBlctSirfrig, cdsldiStifcjfl/sftelcfcSWno, tEntoy^'empIateenlTy} 
*(foMetadataChanoeCt&itiy:TeniptBtB&itry. tMdiTBmplateMatadata) 
-updateTe)iplate(staodld;String, cdsldrString, sHeldrString, tErTtTy:T6mplatBEMry3 
KterremplatfiChai)0B(tEnliy:TenplatBEn1iy.tKk^^ ^ 
-updataBoOiCatagftldrSMng. pdaMSlriha, dtBlctString. t&myfTemplatBEmryi 



'fcm 



Cache 


iMan^gsr 




•fnvaIldaiaOrRegenEachln«tBnoe(tM 


diTeiTiplBlsMeta 


tdBta, inBtanc8B:Ust) 



-$t8mplataDelBte(tMdbTenvlBleMetBdata) - 
-$tsiT^lataMetadalBUpdBte(tMd:Ternptat0MBttd^^ 
-$i»inplBtBUpd8lfi<1Md:TeniptataM8tadata) 
«gtefnplateAndM8tedataUpdatetlMd:TBfliplatBMBtadata) 



i)oncySsy8ReoanerBto{lpHMd:TernplateMetadata, path:Sii1i«} : bdolean 
-regsneratsOMdiTeniplaleMetadBta* raganlnstaiicaldJnq ' 
OBBneTategMdiTemPtoteMrtadata. lMd:tnsianoeMaiadala) 
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The following pseudo-code shows how the metadata-tasg is handled: 

processMetadataMsg(httpBody : InputStream, contentLength : int) 
// Read the msg, then vrnmarshal it into a castor msg object 
bytel) buffer new byte [contentLength] 
int bytesRead =' httpBody. read (buffer) 
String string - new String (buffer, "UTF-S") 
StringReader str^-ngReader = new StringReader (string) 
metadataUsg = Metadata]fiI$g.uninarshalMetadataMsg( StringReader) 

String stageld = metadataMsg. getstageld * * 

String cdsid = metadataMsg. getCdsId 

String siteld s metadataMsg. getsiteld * 
for each tencplateEntry in metadataMsg -getTejnplateEatry, 
if ten^lateEntry.getTemplateEvent == add, 

addTeznplate( stageld, cdsId, siteld, metadataMsg. getDald, ten^lateEntry) 
else if teinplateSntry.getTemplateBvent == delete, 

deleteTemplateC stageld; cdsId, siteld, teziiplateEntry) 
else if t eiiplateEn try. SetTen^lat ejrvent ,== update-metadata, 

updateTeniplateMeta5jata(stageid, cdsXd, siteld, teniplateEntiy) . 
else if templateEntry . getTemplateEvent == update-template, ■ ' 

updateTemplate( stageld, cdsId, siteld, templateEntry) 
else if templateEntry. getTeraplateBvent == update-both, 

updateBoth ( stageld, cdsId, siteld, templateEntry) 

addTenplate (stageld : String, cdsId : String,' siteld : String, dald : String, 
tEntry : Ten^lateEntry) . • 

// Begin transaction 
' mdCon. setAutocommit (false) 

// Ignore the metadata-msg if it's redundant 

if mdCon.teraplateExists (stageld, cdsId, siteld, tEntry -get Pa thAt Pg ) , return 

insert Tentplate metadata row tMd (and rows in associated tables like 
ModuIeParameter and TemplateState) given the values in tEntry 

delete WebServerLookup rows, with template's stageld, cdsId, siteld, and pathAtPg 

// End transaction 
-mdCon . comndt 

. mdCon . setAutocommit ( true) 
Peri odicExpirat ion . templateAdd { tMd) ■ 

dele teTemplate( stageld : String, cdsId : String, siteld : String, 
tEntry : TemplateEntry) 
// Begin transaction 
mdCon . setAutocommit {false) 
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String pathAtPg = tEntry;getPathAtPg 

tMd « indCon;getTeinplateMetadata{stageId, cdsid, siteld, pathAtPg) 

// Ignore the itietadata-msg if it's redundant . . 
if tMd -= null, return 

if tMd.getObjectrd 1= tEntry.getTemplateMetadata.getObjectld, return 

invalidateAllInstances (tMd) 
mdCon . deleteTenplate ( tMd) 

delete WebServerLookup rows with template's stageld, cdsid, siteld, and pathAtPg 

// End traiisaction 
indCon . coiranit 

ndCon. setAutoccnnmit (true) 
PeriodicEbqpiration. templateDelete (tMd) 



invalidateAllInstances (tMd : Templat Metadata) 

instances = mdCon.findlnstances (tMd) ... 
for each iMd in instances, ^ . . * • 

iMd. setlnstanceState (Invalid) 

iMd.setE3cpirationTiinestairip(new Date) 

invalidateEachlnstance (instances : List) 
. for each iMd in instances, 

mdCon.setTemplatelnstanceStatedMd, Invalid) 

U]?daterenplateMetadata(stageId : .String, cdsid : String, siteld : String, ' • 

tJSntry : Templat^Entry) 
tMd = indCon.getTeinplateMetadata{stageld, cdsId, siteld, tEntry. get PathAtPg) 

// Ignore the metadata-msg if it's riedundant 
if tMd null, return 

if tMd.getObjectId != tEntry. getTentplateMetadata.getObjectld, return 

if tMd.getModCount ->= tEntry.getTenplateMetadata.getModCount, return • 

if tMd.getMetadataftash tEntry.getTeinplat^et'adata.getMetadataHash, return 

doMetadataChange(tEritry, tMd) 

// Called when a metadata-msg indicates module metadata changed- * 

// Compares module parameters from templat is metadata to those in metadata-msg and 

// invalidates instances according to the fpllowing rules: , 

// if any parameters were added, 

// ' -> invalidate (no xegen) all instances 

// if any sensitive or dependent parameters were, removed, . 

f/ -> invalidate (no regen) instances with ModuleValues for- those parameters 

f/ if any no-cache parameters were removed, 
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// just update metadata 

II 

II Note: here's an optimization that could he made to the case where sensitive or 
// dependent parameters are removed. Instead of deleting an instance x that uses a 
// removed parameter P, Y_paramHash = X_paramHash - P. If Y^aramHash* is not in 
// the instance metadata, add it and reiiame X's cache file from Xj)aramHash^v.ext 
// to Y_j)arajnHash_v.ext . 

doHetadataChange(tSntxy : TemplateEntry, tMd : TemplateMetadata) 

// Begin transaction • 
mdCon . setAutocommit ( false) 

// Invalidate all instances if any parameters were added. 
paramAdded » false 

for each nParam in tEntry.getTei^platelSetadata.getModuleParams; 
f o\md = false 

for each oParam in tMd.getModulePaxameters, 
' if nParam-getName == oParam.getName 

. nParam.getOptions == oParam.getOptions, 
found = true * • 
break _ 

if . 1 found, ■ . j: " ■ 

paramAdded = true 
break 

if !, paramAdded, ' * * , ' 

// Invalidate instances that use any sensitive or dependent parameters that 
// were removed. 

for each oParam in tMd.getMdduleParameters, 
found = false 

for each nParam in tEntry.getTemplateMetadata.getHoduleParams, 
if nParam. getName == oParam :'getilame 

&& nParam. getQpt ions ~ oParam. getOptions, 
found - true 
break • . 
. if 1 found && oParam. getType !« no-cache, * • 
removedParamLi St . add (oParam) 
if 1 removedParamList.isEmpty, ^ ' 

instances = mdCon.getInstancesUsingParam8(tMd, removedParftmList) 

update Template metadata row (and rows in associated tables like ModuleParameter 
and TempIateState) given the values in tBntry 

// End transaction 
mdCon. commit 

mdCon. setAutocommit (true) " • " 

if invalidateAlX, 

invalidateAllInstances (tKd) - ' * 

el»e. if a.nstances. isEmpty, 

inyalidateEachlnstance ( instances ) 



16 



Vignette Corporation ConfidentiaJ 



0 -■ : 6 . 

PeriodicExpiration . teinplateMetadataUpdate ( tMd) 

updat€iTemplate(stageId : String, cdsid : String, siteld : String, 
t Entry : TemplateEntry) 
tMd = indCon.getTeirrplateMetadata(stageId, cdsId, siteld, pathAtPg) 

// Ignore the ilnetadata-insg if it's redundant 
if tMd-~ null, jretum 

if tMd.getObjectId != tEntry .getTeraplateMetadata.getObjectId, return 
if tMd.getModCount >= indMsg.getModCount, return 
if tMd.getTemplateHash == tHash, return 

doTemplateChange (tEntry, tMd) 



doTemplateChangeitEntry : TeznplateEntry, tMd : . TemplateMetadata) 
String pathAtPg,= tEntzy.getPathAtPg 

// Begin transaction 
mdCon, setAutocoinmit (f alsg) 

■ r - • ■ • 

tHash = tEntry. getTemplateMetadata.getTemplateftash 

tMd. setTemplateHash{ tHash) ■.. - 

tState 5= tMd. getTearplateState( tEntry .getDald)' 
.if tState'== niill, 

tState = new TemplateState • ^ 

.tState-setDeployinentAgentld{ tEntry. getDald) ' .'• 

te^TprateStates = tMd. getTentplat estates 
templateStates . add (t State) 
•tState.setT6niplateHash{ tHash) . 

tMd*setObjectId(tEntry,getTeinplateWetadatajget:ObjectId) 
tMd. setModCouixt ( tEntry, getTenpla teMetadata . getModCount ) 
tMd. update 

// Get this- ten5)late's valid instances. Skip regen instances, since they've 
// already been been cleared by some other event. If PG's already doing the • 
// regen, the placement ten^plateHash vilX control whether it's placed or regen 'd 
// again. , 

instances = indCon.getAllInstances(tMd) . . ' 

// End transaction 
mdCon* commit 
. mdCon.setAutocommit (true). 

cm.invalidateOrRegenEachInstance(t24d, instances) 

updateBoth(stageId ; String, cdsId : String, siteld : String, ' . 

tEntry-.: TeinplateEntry) 
tMd = mdCon.getTemplateMetadatatstageld, cdsId, siteld, tEntry. get PathAtPg) 
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// Ignore the jnetadata-msg if it's reciundant- 
if tMd == null, return 

if tMd.getObjectId .1= tEntryigetTeinplateMetadata.getbbjectId, return 
if tMd.getModCoiant >= mdMsg.getModCount, return 

if tUd.getMetadataHash 1= tBntry.getTeniplateMetadata.getMetadataHash, return 
doMetadataChange(tBatiy, tMd) 

if tHd.getTeznplateHaah 2= tE^itry.getTenplat Metadata. getTemplateHash, retuzn 
doTemplateChangeCtSntry, tMd) 
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4.2 RecordUpdateMsgHandler 

Handles the AM-CM i^ord-update-msg described on page 12. 

The following diagram shows the classes that process the record-update-insg: 



RecordUpdateMsgHandler 



4-processRecordUpdateMsg(httpBody:lnputSlream, contentLengthJnt) 
>updateRecordState(8tag.eid:String. odsld:String. guidiStHng) 



-cm 



CacheManager 



-invalidateOrR8genEachlnstanoe(instances:Ust) 



Regenerator 



-poIfcySaysRegenerate(tpltMd:TernpIateMetadata. pathrStririg) : boolean 
-regenerateCtWdiTemplateMetadata. regenlnstanceld:int) 
-regehBfate(tMd:TemplaleMetadata. iMdrlnstanceMetadalal ' 
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The following pseudo-code shows how the record-update-msg is handled: 

processRecordUp^[ateMsg(httpBo<^ : InputStream, contentLength : int) 

// Read the msg, then immarehal it ijito a castor msg object 
. 'byte[] buffer = new byteEcoiitfentLength] • 
int bytesRead = httpBody. read (buffer) 
■String string = ijew String(buf f er, "UTPrS^') 
StringReader stringReader = new StrijigReader (string) 

ruMsg = RecordUpdateMsg.miinarshalRecordl3lpdateMsg( StringReader) . ' . 

Strang stageld ^ ruMsg.getStageld " ^ 

String cdsid = ruMsg.getCdsId 

// Begin transaction 
. mdCdn /setAutoconttnit ( false ). 

. // (Set the valid, instances affected by the updated and deleted repords. 
. // Skip regen instances, since, if they're still waiting to go to PG, they'll get 
' // the new data. If PG's already doing the regen, the placement 
// record-state-giiid will control whethef it's placed or regen' d again, 
instances s= mdCpn.getInstancesfiyCoiitentTd(stageId, cdsldv • 
ruMisg . enumerateRecordld) 

updateRecordState'(stageId, cdsId, ruMsg.getGiiid) *- 

. // End transaction 

mdepb. commit ... 
mdCon . setAutocommi t { true } 

cm . iiivalida t eOrRegenEachlns tance ( instances } 

tipdateRecordState (stageld. : String, cdsId : String, giiid : String) 

RecordState rcdState ■= mdCon.findRecordState (stageld, cdald) • . . .1.. 

rcdState.setGuidCguidj 

mdCon . update ( rcdState ) 

catch (RecordBoesNotExistException .e) 
rcdState = new RecordState. 
rcdState. set Stageld (stageld] ' ' 

rcdState . 5etCdsId(cds Id) 
rcdState - setGuid{guid) 
mdCon. insert (rcdState) 
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The following diagram shows the classes that process the placement-msg: 



PtacennentMsgHancller 



+processPIacement(httpBody:lnputStream, boundflry;Str1ng) 

-pIace(pcmtM8g:Plac9mentMsQ, cachecJTpttCachedTemplate, hitpBody:lnputStream, boundary:8tring. outerUriA1Ws:String6uffer) 
-insertWebsen/erti>okup(tMd:TempiatBMfltecteta, pcmtMsQiPtecementMsg, cachedTptfcCachedTemplate) 



-cm 



CacheManager 


-cacheRoot: String" 




^cheRIeExtensionSsi : String 




-cacheFilaExtensionNoSsi : String 





•getFu1iFnename(stagelc}:String, cdsldrString, 8iteld:Stnng, pathAtPg:Stnng, paramHash:String, versioniint, extansionzString) 

-fletC)uipiitStream(path:String) lOutputStreanfi . ' . 

rwrttBSsi{stream:OutputStream, outputOffsetint) 

-wrfteSsl(8tream:0iJtputStream, outputOjffsetint, componentPatl^tPg:Str1ng) 

-dos80tJtputStrearii(streanri:OijlputStrBam) 

-de!8teFiIe{path:String) 

-p6licy8aysReg8nerate(tMd:TemplateMetadata, path:String} rboolean . - 
-r8genefate(tMd:TeniplateMetad&ta, instancel.d:int) : * • . 

-regBneratB(tMd;TempiateMetadata, iMd:lnstanceMetadata) 



Regenerator 



-pollcySaysRageneratettpItMdrTemplateMetadata, pBth:Strfr1g) : boolean 
-resenerate(tMd:TempleteMetadata, regenlnstanceldrint) 
-r9generate(tMd:TemplateMetadata, iMd:instanceMetadata) 

The following pseudo-code shows how the placement-jnsg is handled: 
// httpBody points at beginning, of boc^ 

prbcessPlacement (httpBody : Inputs ta:eain, boundary : String) 
size = getContentLeiigthOfNextBart (httpBody, boxjuidary) 
read size bytes into buffer . 
stringReader = new StringReader (buffer, 
msg - FlaceinentMsg.iinznarshalPlacement:Hsg{ StringReader) 
outerUrlAtWs = null 

. - if xnBg.hasDynajnicOuterTeniplate, 

outer = msg.getDynamicOuterTeinplate 
outerUrlAtWs = outer .getUrlAtWs 



// Begin transaction 



e 



outer msg.getDynaxnicOuterTeznplate 
outerUrlAtWs = outer. getUrlAtWs 

// Begin transaction 

mdCon.setAutocQiranit (false) • . 

insert = true 

// If another placement-iosg concurrently updated the table., don't insert . 
if mdCon . hasWebServerLoolcup ( outer . getUrlAtWs ) , 

insert' = false 

// If a concurrent template add changed the template to cacheable, 
// don't insert 

• if mdCon.hasTeii?>lateMetadata(msg.getStageId, msg.getCdsld, msg.getSiteld, 
outer . get Pa tliAtPg) 
insert s false 

if insert, 

WebServerLoo)aip wsLookup = new WebServerriOo3a:p 
wsLoo3cup . setUrlAtWs { outer . getUrlAtWs ) 

wsLookup-setTeinplateKey(null) ^ * • ' • 

wsLooki^ . setStageld (msg . getStageld) 

wsLookup.setCdsId(msg.getCdsId) ■ • , 

wsLookup . set Si teld {misg . getSi teld) 

wsLookup . setPathAtPg,( oute? . getPath^ 

mdCon. ihsertWebServerLookup (wpLookup) • 

// End transaction . - , 

mdCon • coiramt 

m(iCon. setAutocommit ( true) 

.for each cachedTemplate in placemeritMsg, 

place (msg, cachedTemplate, httpBody, . boundary , outerUrlAtWs) 



// Upon exit, next byte in httpBody. is the data in thfe next part 
getContentLengthOfNextPart (httpBody : InputStream, boundary) : int 
read CRLF 

read — boundary CRLF 

read "Content-length:* 
"read length CRLF 
."parse the length 

eat the CRLF that preceeds the bpdy-part 

return the length 

/ / httpBody points at or just prior to next boundary 

place (placeroentMsg : PlacementMgg, cachedTplt : CachedTemplate, 

httpBody : InputStream, boundary : String, outerUrlAtWs : String) 
:*"*'"* * 

// If this is the outer cached page, remember the outer URL at WS to save in 
// included component's inst MD. Do this 1st to get the ti*rl@Ws even if the 
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// outer teinplate is ignored below. 

if cachedTplt.getUrlAtWs 1= null cachedTplt^getUrlAtWs. length 1= 0, 
outerUrlAtWs . replace (0 , Integer .MAX^VALUE , cachedTplt . getUrlAtWs ) 

// Ignore the placement if CM processed a delete template wjiile or after PG 
// generated the page. 

Metadata tpltKd = mdCon^getTeinplateMetadata (placementMsg.getStageld, 

placementMsg.getCds.Id, placeinentMsg.getSiteld, cachedTplt .getPathAtPg) 
if tpltMd « null, 
return 

// Ignore the placement if CM processed a inetadata deploymentc after PG generated 
//. the page. This can occur when a PG generates a placement while the template 
// metadata is being updated. 

if tpltMd. me tadataHash 1= cachedTplt imetadataHash, 
return 

// Ignore the placement if CM processed a ten^late deployment after PG generated 
// the page. This can occur when a PG g^erates a placement while the template 
// is being updated, '* . 

if tpltMd. templateHash cachedTplt .templateKash, 

. throw OldPlacementExcesption . • • 

// Ignore the placement if CM processed a cleeur cache after P6 gdierated the 
// page. Thi9 can occur when 1 PG processes clear cache while another generates 
//a placement for an instance being cleared.* 
if tpltMd- clearCacheld ! = . cacheda^plt-ciearCacheld, 
throw OldPlacementException 

// Ignore the placement if CM processed a record deployment after PG generated 
// the page. This can occur when a PG generates a placement while a record 
//is being updated. 

RecordState rcdState = mdCoh.getRecordState(piacementMsg.getStageId, 

placementMsg . getcdsid) 
if rcdState . getGuid != cachedTplt .getRecordStateGuid, 
throw OldPlacementExjception * 

// If a regeneratable expiration occurred during a regenerate, 
// restart the regeneration, 
catch OldPlacementException 

if cachedTplt -hasRegenlnstamceld, 
- cm. regenerate (tMd, cachedTplt. getRegenlnstanceld) 

return 

// Ignore the placement if it's a duplicate. This occurs when 2 PGs process 
// concurrent requests for the same instance. It also occurs at preexisting 
// caches when a new WS or PG cache is added to grow system: capabilities. 
Instance maxinst = mdCon. getMaxVersionOf Instance (tpltMd. getKey, 

cachedTplt . getParamHash) 
if maxinst !- null && maxinst. state ^s^. Valid, 

// PlacementMsgHandler needs to handle wsLookup info even if . instance is 

// already cached (since there can be multiple url@ws per teinplate, and since 
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// WS can be added after instances are. cached without parentUrieWs values) 

/ / Zt this is the outer cached page, 
if cachedTplt.hasUrlAtWs,. 

// Start a metadata db transaction 
mdCon . setAutoCommit ( false) 

insertWebServerLookup{tpltMd, placementMsg, cachedTplt) 
maxlnst . setOuterUrlAtWs { cachedO^lt . gtetUrlAtWs ) 
mdCon.updatelnstance (maxlnst) 

// End transaction 
mdCon.co^oimit 

KtdCon . setAutocommi t ( true ) 

else // cdznponent or direct P6 request 

if outertTrlAtWs != null, ■ • 

jnaxins t . setOut erUrlAtWs ( outerUrlAtWs ) 
mdCon.updatelnstance (maxlnst) v 

return 

//If there's instance metadata/ increment its version 
•version = 0- . , .• .• ' . . . • 

if maxlnst .1= null, . . - . 

version- = maxlnst. getVersion + 1 

// •Determine the cache file. extension . 
'if .cachedl^lt.getlncludeCount > 0, 
. ext cm.cacheFileExtensionSsi 
. else ■ • . ' ff ' r ■ . ' 

ext cjn.cacheFileExtensioriNoSsi 

// Build the cache filename: cacheRoot/stage/cds/site/pathAtPg/parainHash__v.ext 
path = cm.getPullPilename{stageid, cdsid, siteld, cacheTplt .getPathAtPg, 
cachedl^plt.getParainHash, version; ext) . 

// See if getoutputstream can use java.nio. channels. FileCSiannel lock methods to 
// only allow 1 thread to open the file at a time. 

stream = cm.getOutputStream(placementMsg.stageId, placementMsg.cdsId, 
placemen tMsg.getSiteld, path) 

// Abandon the placement if another thread created and locked the file just 
// before I did (see section 14.7,2.1 Problem .1 - concurrent open and write to 

the same file by different threads in the same process) 
catch ( FileLockedExceptibn . e) 
return 

.size = getContentLengthOfNext;Part (httpBody, .boundary), 
inputldx =0 
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InstanceMetadata iMd s= new InstanceMetadata 
List includeMdList = iMd. getlncludes 
for each include in cachedTplt, 

// Write the template output preceeding the include 
while (inputldx < include. offset) 

byte[] buffer = new byte [include, offset - inputldx] 
bytesRead = httpBody. read (buffer, 0, buffer, length) 
. if bytesRe^d* -1 . . * 

close stream; delete cache file;. log error 
throw PreinatureEOF 
inputldx +c bytesRead 

stream.write (buffer, 0, bytesRead) 

// Remember the data (including the output offset) for .each SSI written. 
. // Store the ssi data in the instance metadata so the filter can skip 
// directly to the appropriate spot in the cache file during delivery rather 
// than having to parse the entire file. 
isComponentinclude = false 
component Path = null _ 

Include includeMd = new Include ' . * ' 

includeMd i setHandler ( include , getHandler) 
includeMd.setOf f set (includis.of fset) 
List ipList = includeMd. getlncludeParameters 
for each param in .include.^ . ' . 

IncludeParameter ip = new IncJudeParameter 

ip.setName( param. getName) ^ % y • . . . 

ip.setValiie (param. getValue) . . * 

• ipLiat.add(ip) . * ' 
» ' ' ^.^ ' ' ' 

// cod include handler's component information is represented in the ' 
// placement *msg as follows: . ' ' 

• // • ' . 
// <include handler=*cod" off set="100'>. 

// <param name="type" value=" component "/> 

// <param names="path-at-pg" value="/hostAbc/x. jsp" /> 

// </include> 

if include. getHandler == *.cod*, 

if param. getName ~= *type*. && param. getValue » 'component*; 

isComponentlnclude true 
else if param. getName =- *path-at-pg' , 
component Path » param. getValue 

CQzrtponentSsi » isCon^onentlnclude && coznponentPath I* null ' 

// Write the SSI . . 
ssiLength = componentSsi ? cm. writeSsi (stream, include, of f set , compoiaentPath) 

cm. writeSsi (stream, include.of f set) . 

includeMd. setLength (ssiLength) 
includeMdLi s t . add ( includeMd ) 
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// Write the template output that follows the last coinponent 
while (inputldx < size) 

bytef] buffer = new byteEsize - inputldx] - 
l]ytesRead = httpBody. read {buffer, 6, buffer, length) 
if bytesRead ' -1 * . , 

close stream; delete cache file; log error 
throw PrematureEOP 
inputldx += bytesRead 

stream, write (buffer, 0, bytesRead) 

cm. closeOutputStreaiR ( stream) 

// Start a metadata db transaction 
mdCon. setAutoCommit (false) 

iMd . set State ( Ins tanceState . VALID ) 

//If another thread processed a metadata deployment while this thread wrote the 
// cache file, 

tpltMd « indCQn..getTeinplateMetadata(placementMsg,getSt£^geId, 

placementMsg.getCdsId, placementMs^.getSiteld, cachedlt>lt.getPathAtPg) 
rcdState = mdCon . getRecordState (placementMsg . ge tStageld, placementMsg . getCdsld) 
if . tpltMd == null |j tpltMd. m^tadataHash != cachedltlt-metadataHash, 

.// Then th6 cacshe file is invalid (no regen foa: metadata change) . 
// Delete the file and abort the. placement, 
cm. deleteFile (placementMsg- stageld, placementMsg. cdsid, 
placementMsg. getSiteld, path) 

return 

// If another thread processed any of the jEollowing eventp while this thread* 

// wrote the cache file: 

// *' template deployment. 

// * clear cache 

// * record deployment 

else if tpltMd. templateHash 1= cachedT^lt . templateHash |1 
tpltMd.clearCacheld .1= cachedTplt .clearCacheld || 
rcdState,getGuid 1= cachedltlt.getRecordStateGuid, 

// Then the cached file already -expired,. 

//If the instance can be regenerated, 
if cm.policySaysRegerierate{ tpltMd, path), 

» 

// Set its state to regen. 
iMd. setState ( Ins tanceState .REGEN) 
•» else • 

// Otherwise, delete the file & abort the placement, 
cm. deleteFile (placemen tMsg.stageld, placementMsg. cdsid, 
placementMsg. getSiteld, path) 

return 
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// Write inst and ws lookup metadata 



iMd, setTemplateKey ( tpltMd. getKey) 

Did. setParaxnHash (cachedTplt . getParaxnHash) 

iMd.setVersion (version) 

iMd.setRequestContentType (placementMsg.getRespcnseContentType) 

iMd . setResponseCoritentTVpe (placemen tMsg . getResponseCont entType ) 

iMd . setReqaestMefhod { placementlilsg . getReques tMethod) 

iMd , setRegenHost (placementMsg . getHost ) 

iMd. setlsComponent (cachedTplt - getlsComponent) 

iliSd. setBxtension ( ext ) 

stdModuleValues « IMd-getModuleValues 

for each plModuleValues in cachedTplt . eniimerateModuleValues , 
set indModuleValues from plModuleValues 

List idiiist = iMd.getContentTrackinglds 
for each id in cachedTplt .getContentTracking.getlds, 
idList.add(id) 

List actionList ^ iMd. getAct ions . r . . 

•for. each action in cachedTplt . getActipns , . 
Action actionMd = new Action - 
actionMd.setHandler (action. getHandler) 
List. actionMdParamList = a.ctipnMd^getPajrariieters 
•for each param in action .'getPara^is; : . , 
ActionParaineter apMd = new ActibnParajpeter • 
. apMd.setName (param. getName) - 
,apHd ► setValue (param - getValue) 
actionMdParaimList • add ( apMd) 

//If this is the outer cached jpiaga, - . ' 

if cachedTplt .hasUrlAtWs, 

insertWebServerLookup (tpltMd, placementMsg, cachedTplt) 
iMd . setOuterUrlAtWs ( cachedTplt . getUrlAtWs ) 

else // component or direct PG request 

iMd. setOuterUrlAtWs (outerUrlAtWs) 

mdCon . insertlns tance ( iMd) 

// End transaction 
mdCon. commit 

mdCon • setAutoconmiit ( tame) 

//If another -thread processed an event thfit caused the instance we just wrote * 
//to expire, and if the instance should be regenerated, 
if iMd.getState ~ Instance .REGEN, 

// Send regenerate to WS 
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cm. regenerate ( tpltMd, i)fd} 



// Call with autocommit false 

insertWebServerLookup ( tMd : TenplateMetadata, placementMeg: PlaqementMsg, 

cachedTplt : CachedTeiiplate) 

//If another placement-msg concurrently Updated the table, jnst quit 
«if aidCon.hasWebServerLookupicachedTplt.getUrlAtWs) , 
return 

WebServerLookup wsLoolcup = new Webserver IiOOkup 
wsLookup.. setUrlAtWs ( cachedTplt . ge tUr lAtWs ) . ■ 
wsLookup . setTeniplateKey ( tMd . getKey) 
wsLoo3cup, setS.tageld (placementMsg, getStageld) 
wsLookup . setCdsId (placementMsg . getCdsId) 
wsLookup. setSiteld (placementMsg . getSiteld) 
wsLookup . setPathAtPg ( tMd . get PathAtPg) 
mdcon. insertWebServerLookup (wsIiOo)cup) 
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4.4 ClearCacheMsgHandler 

HancDes the PG-CM clear-cache-msg described on page 12, 

The following diagram shows the classes that proqess the clear-cache-msg: 

ClearCacheMsgHandler 



^i)rocessClearCacheMsg(ht^>Body:l^pu^tream,opnte^^^^ 



CacheManager 



•frwalidateOrFtegenEachlnstanc6(tMd:Temp]ateMetadata,?nstEvices:U 



Regenerator 



-policySaysRegenerale(tpllMd:TenniDlat©Metadata, path:Slring) ; Ijoolean 
-regeneralB(tMd:Templat9Metadata, regenfnstancefdiint] 
-regenerate(tMd:TemplateMetadata,.iMd:lnstanceMetadata) 
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The following pseudo-code shows how the clear-cache-msg is handled: 



processClearCacheMsg(httpBoetir : InputStream, contentLength. : int) 

// Read the mag, then unmarshal it into a castor msg object ^ 
byte[] buffer = new byte [contentLength] 
int bytesRead = httpBody. read (buffer) 
String string = new String (buffer, "UTF-S") 
.StringReader stringReader » new StringReader( string) 
ccMsg = .ClearCacheMsg.umnarshalClearCacheMsg{ StringReader) • 

String sf:ageld = ccMsg. getStageld 
String cdsid = ccMsg.getCdsId 
String siteld = ccMsg.getSiteld 

// Get the templates that match, the paths argument 

teitplates ittdCon.findTentplatesfstageld, cdsId, siteld, ccMsg. get Paths, 
ccMsg. getClearCacheOption) 

for each tMd in ten5)lates, 

// Begin transaction r • ^ ■ 

mdCon . setAutocmnnit { false) 

. tMd ■ setClearCacheld ( ccMsg. getClearCacheld) 
tMd-npdate 

// Convert the castor match-map .& match-list objects into the 
// parameters Map defined in the COD Clear Cache API. This means 
// if a match-map and a match-list, have the same typ^, wrap them 
// in a single list since you can't have 2 outer Map entries' with 
// the same module name. 

// Get the valid instances to be cleared. Skip, regen instances, since 
// they've already been been cleared by some other event. If PG's already 
// doing the regen, the placCTient clear-cache-id will control whether it's 
// placed or regen 'd. again. 

instances » mdCdn.findlnstances (tMd, ccMsg.getMatchMap, ccMsg.getMatiliList; 

// End transaction 
mdCon. commit • . 

mdCon.setAutocominit (true) . ' 

cm . invalidateOrKegenEachlnstance ( tMd, instances ) 



33 



Vignette Corporation Confidential 



4 



o 



o 



5 Cache Manager 

The following diagram shows the CacheManager class data members and methods: 



CacheManager 

-host : String 

-port:mt . ' . . ' 

-cacheRool : String 

-cacheRleExtensionSsi : String 

-cacheFIIeExtensionNoSsI : String 

-componentSsiPrefix : String 

-oomponentFonwardPath : String 

■modules : Map ;__ ' ■ 

-getFuI!Rlename(stageld:String. cdsld:String, srt9ld:String, pathAtPgrString, panamHashiString, veratonrlnt, «cten8ron:String) 
•9etOutputStreBm(path:String) : OutputStream 

-writeSsi(8tream:OiJrtpLitStream,outputOfi^et:irit) . . . ' 

-writeSsi(stream:OutpiitStreani| outputOffsetint, componentPathAtPg:String) 
-doseOutputStreamtstreamrOutputStream) 

<leleteRIe(path:Stri.ng) ' ^ ^ , . 

-poli<^SaysRegenerate(tMd:TeniplateMetadata, iMcf:InstancBMetadata) : boolean - - 

-policySaysRegeneratB(tMd:Temp!ateMetadata, path:String) : boolean* . . : 

-regenerate(tMd:TemplateMetadata, regenlnstanceldJnt) ' , • . . 
-regenerate(tMd:TemplateMetadatat iMdrlnstanceMetadata)' : . ' 4 

-invalidateOrRegenEachlnstanae(tMd:TennpIateMetadata, instancesSUst) ' ' ^ " • 

-lnyalidatGOrR&genlnstance(tMd:T8mplateMBtadata, iMd:ln$tanc . " 

-InvandateOrRejaenEachlnstanceOnstancearUst) " ' • ' 

The following pseudo-code describes the CacheManager methods: 

CacheManager. ihit(config : ConfigThingy) ' - 

// Initialize my config variables 
conf ig.get (cacheRoot, CACHE_jiobT) 

conf ig.get (coinponentSsiPrefix, COMPQNENT.SSll^HEPTX) 
cacheFileExtensionSsi =. ''shmtl" 

config.get(cachePileExtenaionSsi\ CACHE_FIL?_EXTENSIQN_SSI) 
cacheFileExtensionNoSsi = ^vgn' 

config. get (cachGFileExtensionNoSsi, CACHE_FXLE_EXTENSI0K1LN0_SSI) 

Based on config info, load each module and invoke its init routine 
Store references to tJie modules in the. modules Map data member 

gecFullFilename(stageId, cdsid, siteld, pathAcPg, paramHasJa, version, ext) • 
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return cacheRoot + + stageld + V" + cdsid + V" + siteld + V" 

+ pathAtPg. + V" + paranjHash + +. String.valueOf (version % 10) + +' 
ext 



getOutputStreamCpath *: String) : OutputStream 

Try to use java.iuo.chajuiels.FileChannel lock methods to only allow 1 thread to open the file at a 
tune. Ideally, threads not getting the lock should letum unmediately or throw an exception. The 
placement would then be abandoned immediately. If jaya.nio can't do that either, then we may 
need to use INI (see section 14.7,2. 1 Problem 1 - concurrent open and write to the same file by 
different threads in the same process).. 

writeSsi (stream : OutputStream, offset : int). : int 
// Build the ssi 

String ssi « *<J~#" + componentSsiPrefix + '\'/' + *?vgnld=* + offset + .•\-~>» 

// Convert it to bytes (should be ASCII, but utf-8 won't hiart) 
Byte[] bytes ssi.getBytes (^UTF-S") 

r ^ • . ■ 

// Write the ssi as bytes,, then return the # bytes written 
stream. write (bytes) 
return bytes , length 

writeSsi (stream : OutputStream, offset*: int, con^onentPathAtPg : String) : .int 
// Build the ssi 

String ssi = + componentSsiPrefix + ^\*' + componentPathAtPg ■ ' 

+ "?vgnld='.+ offset + '^X* — >• 

// Convert it to bytes (should be ASCII, but iitf-8 doesn't hurt) 
Bytet] bytes-= ssi,getByte.s (*UTF-8") » 

// Write the ssi as bytes, then return the # bytes written 
stream. write (bytes) 
return bytes , length 

closeOutputStream( stream : . Outputistream) 



deleteFile (path String) 



olicySaysRegenerateCtMd : TemplateMetadata, iMd : InstanceMetadata) : booleail 
•path = cm.getFullPilename(tMd.getStageId, tMd. getCdsid, tMd.getSiteld, 

tMd.getPathAtPg, iMd.getParamHash, iMd. getiVersion, iMd.getBxtension) 
return regenerator .policySaysRegenerate(tpl tMd, path) 
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policySaysRegenerate{tpltMd - : TemplateMetadata, path : String) : boolean 
return regenerator,policySaysRegenerate(tpltMd, path) 



regenerate { tMd : TentplatjeMetadata-, regenlnstanceld : int) 
regenerator . regenerate ( tMd, regenlnstanceld) 

regenerate (tMd : TeprplateMetadata, iMd : InstanceMetadata) 
regenerator . regenerate ( tMd , iMd) 

InvalidateOrRegenEachlnstanceitMd : TeinplateMetadata, instances : List) 
for each iMd in instances, 

invalidateOrRegenInstance(tMd, iMd) 



.invalidateOrRegenInstance{tMd : TexnplateMetadata , iMd : • InstanceMetadata) 

// Begin transaction 
mdCon. setAutocoinmit (false) 

r ' • . * * 

// Set state to invalid or regen< depending on regenerate policy 
iMd.setState(policySaysRegenerate{tMd, iMd) ? 

InstanceState.REGEN : InstanceState.. INVALID) 

// If the instance is transitioning, to invalid, remember the expiration 
•// timeatainp for the instance deleter, 
if iMd.getState -= InstanceState. INV3U.IP, 
iMd.setExpirationTiinestainp(new Date) . 

// Else if the instance is transiticaiing to regen, send regenerate to WS 

else if iMd.getState == Instance. REGEN, • • ■ 

regenerate (tMd, iMd) 

iMd^update 

// End transaction 
indCon. commit 

mdCon . s^tAatocoinmit ( true) 

invalidateOrRegenEachlnstance (instances : List) 
for each iMd in -instances, 

.tMd s mdCon.findTeinplate(iMd) • invalidateOrRegenlnstanceCtMd, iMd) 
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6 Instance deleter 

The instance deleter is a background thread that wakes up every CLEANUP JPERIOD seconds and 
deletes instances (instance metadata and cache file) whose state has been Invalid for more than 
CLEANUPJ5ELAY seconds. 

Since' CLEAIWPJDELAY is intended to give clients time to receive the Invalid state change before the 
file is deleted, we shouldn't nin into problems deleting the file because someone still has it open. So 
I'm not planning to do anything fancy for file delete. I'll just lise java.io.Hle.delete. The file delete will 
occur before the instance metadata delete. If the file exists, and the delete fails, the instance metadata 
delete will be omitted,.so the file delete will be tried again at the next cleanup. 
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7 Periodic expiration 

The peiiodic expiration thread expires instances according to their expire-policy and expiie-period 
settings. When a template is added or an existing template's metadata changes, its template metadata is 
passed to periodic expiration so it can initialize expiration processing for the template if it's set to expire 
periodically, and if it's not already initialized Periodic expiration keeps a min heap of the next 
expiration for each periodically expired tempilate. The key to the heap is the next expiration time. The 
value is a list of templates whose next expiration time equals the key. When a template is deleted or an 
existing template's metadata changes, its template metadata is passed to periodic expiration so it can 
temoinate expiration processing for the template if it*s not set to expire periodically, and it's cuiientiy 
initialized Aftsx all add/delete/update messages are processed, the thread sleq>s till the isext expire time 
or anoth^ message is received When an expire time occurs, an entry is removed from the heap, and the 
following processing occurs: 

for each tMd in the entry, 

instances = mdCon.getValidlnstances ( tMd) 

tMd. setPeriodicExpirationTimestainp (entry ^getExpireTime) 

indCon-update(tMd) ■ 

cm. invalidateOrRegenBachlnstance ( tMd, inst^ces) 
t - calculateNextE3{pireTiine(tMd). 
addEntry(t, tMd) 
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8 Regenerator 

A Regenerator object is also a thread. The REGENERATION.CONCmREISrCYJLJMrr 
configuration variable is enforced by placing the specified numbw of Regenerators in Hie regenerator 
thread pool. Each Regenerator gets RegenerateMsgs fronj the thread pool input queue. TTie regenerate 
message and queue timeout handlers are described below. 

TTie following diagram shows the classes that process regehOTtion: 



Regenerator 




Cac^eManager 


-fegenHost : String 
-regenPorl : Int 


^componentFopvardPath : String 
-modules : Map 


— > 


^po!icySaysRGgenerate(tpltIVId:Templat0MBtacteta, path:String) : boolean . 
-regenerate([Md:lnstanceMGtadata} 
«-regenerate(tMd:TemplateMetacIata, instanceldint) 
-regenerate(tMd:TeniplateMetadata, iMdrlnstanceMstadata) ^ 
-handteRegenerateMsgtmsgrRegenerateMsg) 
-queueTlnneoutHandler(t]medOutInstances:Ust) 









1,/ 



<<lnterface» 
ICagh&ModMle-. 



-IsRegenSupportedO • boolean 

-filiR8genRequesl(rqst:VgnHttpServleiRequest, insiMdrModuieValues) : t>oolean 



Regenerator, init 

// Initialize iny config variables 
config.get (regenHost, REGENERATEJSOST) 
config. get (regenPort, REGENERATEL.PORT) 

TaekManager . setQueueTimeoutHandler { queueTimeoutHandler ) * 

instances = JndCon. findAllRegenlnstances () 
, for each iMd in instances^ 
regenerate ( iMd) 

The following method is called by other classes to determine whethK- to regenerate: 

policySaysRegenerateJtpltMd : TeitrplateMetada1:a, path : String) 
, if tpltMd.regeneratePolicy Never, 
'■*• return false 
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if tpltMcL.regeneratePolicy == Always, 
return true 

Date now - new Date 

Date lastAccess f Pilelo.IastAccess (path) 

return {now-getTime - lastAccess .getTime) / MILLISEC_PEIUSBC <= 
tpltMd* getRegenerateLastAccessedlnterval 



The 3 following methods are called by other classes to put a RegeherateMsg in the Regenerator thread 
pool's input queue: 

regenerate (tMd : TeinplateWetadata, instanceld : int) 
iMd = zndCon.getlnstance (instanceld) 
regeneratej(tMd, iMd) 



regenerate (iMd : InstanceMetadata) 

tMd = mdCon.f indTeniplate-{iMd.getTeinplateKey). 
regenerate ( tMd, iMd) ^ 



regenerate (tMd : TentplateMetadata, iMd : InstanceMetadata) 

puts a RegenerateMsg in the regenerator queue, with timeout = 

iMd. expirationTimes tanq? + • REGENERATE J4A2L.TIME^TOJREQUEST . 



RegenerateMsg 

tMd : TeinplateMetadata 
iMd : InstanceMetadata 



The following method handles the RegenerateMsg: 

handleHegenerateMqg{regenHsg : RegenerateMsg) 
tMd » regenMsg.tMd 
iMd = regenMsgwiMd 

// Let modules add their parameters to the request 

request = new VgnHttpServletRequest 

for each moduleValues in iMd . enumerateModuleValues / 

module « cm .modules . get (moduleValues , getModuleName) 

if ! jnodule.isRegenSupported, 
return 

if ! module. fillRegenRequest (request, moduleValues) 
return 

// Sec host & port • • 

if regenHost, length ==0, 



40 



Vignette Corporation Confidential 



get host & port from iMd/getRegenHost 
else 

• host = regenHost 
port = regenPort 
request . setServerName (host ) 
request . setServerPort (port) 

•// Set URI to outerUrlAtWs if there's a WS tier, or pathAtPg otherwise 
if iMd.hasOuterUjrlAtWebserver,' 

request . setURI ( iMd . getOuterUrlAtWebserver) 
if iMd.isComponent, 

request . addHeader ( "x-vgn-coniponent-path-at-pg'' , tMdlgetjathAtPg) 
request .addHeader ( *x-vgn-stage-id' , tMd. getStageld) 
request- addHeader C^x-vgn-cds-id", tMd.getCdsId) 
request . addHeader ( 'x-vgn-si te-id' , • tMd . getSi teld) 

else 

request . setDRI {tMd. getPathAtPg) 
if iMd^isConrponent, 

// Add response-conteqjt-type header. /The filter, sets the Content- type of 
// the response before invoking the cfimponent so the PG can transcbde the 
// coiiponent output to match the encoding of the ten^jlate that called the 
// component. 

request , addHeader ( *x:^vgn-respo.nse--content-type'' , iMd.getResponseContentType) 

. // Add Content-type header if a module didn't already do it. 
if request. getContentType i= null && iJJd.getRequestContentType != null^ 
request . setContentType (iMd. getRequestContentType) 

// Add.is-regen header. This header tells the plug^in to not serve the request 
// from the cache.. The header is forwarded in the request to the PG. PG sees 
// it and Jcnows to not serve the request from the cach^. 
request.addHeader (^x-vgn-is-regen", **true*) 

// Add identifiers of different events that can cause regenerations. ' This' helps 
// PG avoid redundant regenerations. 

request . addHeader ( ''x-vgn-teraplate-hash- , tMd. getTemplateHash) 
request. addHeader (*x-vgn-clear-cache-id*, tMd.getClearCacheld) 
request . addHeader ( *x-vgn-periodic-expiration-timestair^" , 
tMd . getPeriodicEa^irationTimes tamp ) 
. rcdState mdCon.findRecordState( tMd. getStageld, tMd.getCdsid) 
" request . addHeader ( •x-vgn-record-state-guid'' , rcdState . getGuid J 

// Add param-hash header so filter doesn't have to confute it unnecessarily, 
request . addHeader ( "x-vgn-param-hash' , iMd . getParamHash) 

send request . • . 

wait for the response with timeout = RBGENERATEL.TIMEOUT 
'if. response received && response. isOk, 
tte.: ^ placementMsg = response. -getPlgcementMsg 

// PlacementMsgHancller checks regenlnstanceld in places where regen placement 
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// differs from normal placement. These cases are caused by race conditions 
// between different types of expiration events on the same instance. ' The 
// races conditions only occur when the teinplate's regenerate-policy causes 
// regeneration. For example, if a clear-caphe occurs for an instance that's 
// already being regenerated due to a template update, the regen may or may 
// not pick up the new clear-cache-id. If the placement handler sees that a. 
// regen placement's clear-cache-id is old, it restarts the regeneration, 
// wherea5 an old normal placement is just discarded. 
placementMsg . ^etRegenlhstanceld (iMd. getKey) 

send placementMsg to PlacementMsgHandler Tas3cMariager 
else if timeout or response failure ~ OldTemplater 
// Timeout or failure - invalidate the instance- 
iMd . setState ( Invalid) 
iMd.setExpirationTimestamp'fnew Date) 
mdCon . updat einstance ( iMd) 

// Else failure, was OldTeniplate ► That means the iPG- that received the regenerate 
// did not have the version of the template that caused the request. Leave the 
// instance as-is, because a subsequent template-update will be received from 
// the AM of the PG that ^returned OldTemplate, after* which requests to that PG 
// will succeed. So worst case, the request caused by the last AM^^s template- 
// update is guaranteed that all PGs have the new version of the template - 

// Handles RegenerateMsgs that time out in the Regenerator queue. 

queueTimeoutHandler (timedOutlnstances : List) for each iMd in timedOutlnstances , . 
iMd . setState (Invalid) 
iMd. update 

8.1 Regenerate request 

This is an HTTP request sent to' the hostrport firom the cKent request that caused the 6iig?hal instance to 
be cached. 

Websphere is the lowest conGmaon denominator. Its Servlet Engine. (SE) taUcs OSB rather than HTTP, so 
CM can*t easily send regenerate requests directiy to a Websphere SE. The solution is to save in the, 
instance naetadata the Host header from the client request that caused the original instance to be cached 
Regenerations are then sent to the saved hostport The REGENERATE_HOST and 
REGENERATEJPORT configuration variables override this behavior by specifying a hostport to 
which aU regenerate requests are sent 
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9 Configiiration Space 

9.1 Variables at tiie CM/cache level 

9.1.1 PubUcvariaWes 

9.1.1.1 HOST 

The name of the machine CM mecutes on. 

9J1.1.2 PORT 
Type -integer 

The port CM Kstens to for HTTP messages firam AM and PG. 
Min-1 . . ■ . 

r . * • . 

Max-65535 

9.1.1.3 CACHEJROOT 

■Type -String . ' ' . . ' 

The fuUyquaJified.pathofthe cache's directory root . .. < 

9.1.1.4 CA.COT_FlLE_Ejn'E3^SI0N^^ • . . J . 

If a cacheable template calls any componeTits, its cache file is saved with this extension. Server-side, 
includes must be enabled at the web server for the specified extension. 

If a cached file does not call any components, it will be saved with the extension from 
CACHEJPILEJ3XTENSl6NJSrO_SSL 

Default - shtml QJS, Apache, and iPlanet automatically do server-side parsing of .shtml files). 

Dyiiamic change - start using the new value. Customer needs to foUow the following procedure when 
changing this variable: 

1. Configure your web servtar to parse the new extension 

2. Change CACHE_FILEJEXTENS10N_SSI to the new extension 
.3. Execute VgnCDS.clearCache(new StringO {7*"}, null) 

4. Configure your web server to not parse the old extension 
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9.1.L5 CACHEJPlLE.EXTENSION_NO_SSI 

If a cacheablc template does not call any components, its cache file will be saved with this extension. 
Server-side includes should not be enabled at the web server for the specified extension. 

If a cached file calls any components, it will be saved with the extension from 
CACHE JTOJBXTENSION^SSL 

Default -vgn. ' . 

Dynamic change - start using the new value. 

9.1.1.6 COMPONENT^SSIJPREFIX 

This config var will go away for one of the following reasons: . 

1 . exec CGI is probably an unacceptable security hole. Jam is researching whether we can mate 
include virtual work for nS. 

2. Even if exec CGI turns out to be ok, the prefix-should be derived from the TYKE {US, iPlanet, 
Apache, etc) of the WSs associated with the cache. 

When a cacheable template calls a component, the component output is extracted from the template's 
output and replaced with a server-side include (SSI) that either retrieves a cached coniponent or sends a 
domponent request to the PG. Thetype of SSI directive depends on the web server type. 

COMPONBNT^SIJPMFIX contains the string between ''<^^^^ 

ForllSweb servers, the default. is exec CGI". 

For other web servers, the default is ''include virtual". 

9.1.1.7 POOL^IZE 
Type-integ^ 

Placements, template changes, record changes, and clear cache commands are all bandied by the same 
thread pool. This configuration variable determines how many threads are in the pool. 

MlTl-1 

Max-.256 ^ 
Default- 10 

9.1.2 Hidden variables 

These valuables conUol inleraal workings that the customer probably doesn't need lo worry about, but 
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we don't want to hard-code. We would like to handle them something like the config variables in V6 
that are added to template files in comments. ITie variables are not automatically created, and they are 
not documented. But if created, code will use them. This allows support or VPS to alter the behavior if 
necessary, but hides unnecessary complexity ftom the customer. - 

9.1J.1 CLEANUPJPEKIOD 
Type -integer 

The instance deleter wakes up every CLEANUP_J>ERIOD seconds to delete invalid instances. 
Min-0 

Max -2592000 (30 days) .... 
Default- 600 ■ . 



9.1A2 CLEANOPJ&ELAY 

Type -integer ^. - . . 

When the instance deleter wakes up, it only deletes instances (instance metadata and cache file) whose 
states have been Invalid for more than CLEANUP_DELAY seconds. The delay makes sure the instance 
metadata has lime to propagate to all cUents he&re the cache fUe is dejtted. Althou^ the propagation 
should occur within a second, the default is conservatively set a couple of orders of inagnitude larger. 

Min-0 ' . ■ ■ ^ 

Max -2592000 (30 days) • . • 

Default-300 



9.2 Variables at the stage. CDS or stage:CDS:sUe level 

For maximum granularity, a separate set of these variables can be defined at each intersection between . 
CDS and site. Or they can bfc defined at the CDS level if the values are the same for aU the sites in a 
CDS. If a variable is specified for a CDS and one of the sites associated with that CDS, the CDS &ite 
value oven-ides the CDS value! 

9.2.1 REGEI^3ERATE_HOST 

nie name of the machine CM sends ncgcneratc requests to. If blank, CM sends each regenerate request 
to flie hostiport that received the request that cached the instance. 
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9.2.2 REGENERATE^PORT 

Type integer 

The port CM sends regenerate requests to. Used when REGENERATE JSOST is not blank. 
Min-1 , 
Max -65535 , . 

923 REGENERATE.CONCURRENCYJLBto 

Type - integer 

The maximum number of concurrent regenerate requests a CM makes to a Page Generator (PG) when 
multiple instances need to be regenerated. 

Min-l ■ 

ilaxrl024 - .. , 

Defaiilt-2 . • . 

9.2.4 REGENERATOJNIAX_TBlE_Tp_REQIJEST ' 

' Type-integer . • ."• • . 

The maximum number of seconds from expiration that an old instance resides in the cache while its 
regenarate request waits to be issued td'a i»G. if the regenerate request cant be sent within-this time, the 
. old instance is invalidated, and the regenerate i^uest js discarded./ 

Min-l • ■ * 

Max -3600 

Default- 300 ■• . ' 

9.2.5 REGENERATE_TIMEOUT 

Type -integer 

The maximum number of seconds GM waits for the regenerate response to a regenerate request If the 
timeout occurs, the old instance is invalidated. 

Min-l 
Max -3600 
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Default-60 
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10 Policy definition file 

expire-policy and regenerate-policy were intentionally represented as enumerations (rather than distinct 
boolesms like ContentChange) to smplify selection of the useM 

10.1 expire-poUcy 

One of the 4 following values: ^ 

1 . ContentChange - yields freshest pages. Template must use content tracking API. 

2. PeriodicIfContentChanges - use if content changes too fast for policy 1. Template must use 
content tracking APL 

3 . Periodic - use if (template does not use content tracking API or uses content not managed by 
VCM) and (content changes on a regular basis). 

4. None - the default. Instance expires when teii^late is updated or cleared via clear cache API 
command. Useif (template does not ijse content tracking API or uses content not managed by 

' VCM) and (content changes under control of another template). The template that initiates the 

content change must use clear cache API to rapire templates that use fte new content. 

'• . » 

102 expire-period 

still heed to tMnk about: • 

• illegal combiniations like day^ofrinbnths:^ or 30 for iP'ebruary 

• daylight savings time changes * . • 

expire-period is specified in the following sub-elements: minute, hour, day-of-monlh, raonth-of-year, 
and day-of-week. Each sub-element is a string conforming to die following syntax: 

* I elementList | everyXUnits ^ 

*= all legal values 

elementList = element, element, .„ 

elemmt = number [ inclusiveRange 

incluMveRange = number-number 

everyXUnits = /number 

Days may be specified 2 ways: day of Uie mondi and day of the week. If both are specified, they ai-e 
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additive. 

The tiiaie zone is the local time zone of the machine running cache fijanager. 
Examples: 











aay-oi-weeK 


result 


0 


8 


* 


* 




every day at 8:00 am 


0,30 


* 


* 


* 




every 30 minutes 


0 


0 


1.15 


* 


1. 


raidnigjit every 1** and 15** and 
every Monday 


12 


* 


* 


* 


1-5 


every 2 minutes Monday-Friday 



10.2.1 mimite (0-59) 

N^nutes of the hour that the template expires on 
Legal values are 0-59 
Default- 0 

10.2.2 hour (0-23) , ^ 

Hours of the day that the template expires on 
Legal values are 0-23 
Default -0 

10.2.3 dayrof-iQOUth (1-31) 

Days of the month that the template expires pn 

Legal values aife I-3I 

Default-* 

10.2.4 mo]ith-of-year(M2) 

Monlhs of the year that the template expires on 

Legal values are 1-12 

Default-* 
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10.23 day^of-week (0-5 with O^Sunday) 

Days of the week that the template expires on 
Legal values are 0-6 with 0=Sunday 
Default-* 

10.3 regenerate-policy 
One of the 3 following values: 

1. Never 

2. Ifl^tAccessedWithinRegenerateLastAccessedlnterval 

3. Always 

10.4 regenerate-last-accessed-interval ^ 
Type - number of seconds 

The interval following the template instance's last access tipie during which expiration causes 
regenetation to occur, • v : • . 

The granularity depends on. the web server type. See the Webserver Mugin de^gn document {11] for 
more information. 
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11 Plug into generic framework 

The cache manager runs as an aglet in the vgnagent framework, vgnagent is.a Vignette managed 
process from config's point of view. For cache managtt', COD conf5g sets up a vgnagent with 2 aglets: 
HttpAglet and CmAglet. The ordinal of CmAglet must be less than that of HttpAglet. In addition, 
config must add the following common.http.action mappings for CmAglet: 

• /placement- com. vignette.cod,cm.PlacementMsgiReceiver 

• /metadata- com.vignette.codcm.MetadataMsgReceiver 

• /recordUpdate - com.vignette.codcm.RecordUpdateMsgJleceiver 

• /clearCache - com.vignettexod.cixx.CleaiCacheMsgReceiyer 

vgnagent calls initialise and start methods on each aglet in ordinal order. It calls the shutdown method 
on each aglet in reverse ordinal order, 

CmAgletinitialize initializes the singleton CacheMan'ager object. Then when Ht^Aglet initialiize 

occurs, and the cm message receivers are initialized, then can get what they need fro 
CacheManager singleto^.- 

ILl Message receivers * , ^" 

•The CM rnessage receivers extend HttpAction and are invoked by the HttpAction infrastructure'-with a - 
reference to an HTTP message. Each receiver invokes its corresponding message h.andler. • 
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12 Monitoring attributes 

Here' s a 1^ cut at the metrics that CM can collect I also.list some collected by WS and PG, because ' 
they seem basic to seeing how caching is perf onning. And some listed as collected by CM could be 
collected by PG during page generation instead. 

Another idea which might replace some of the metrics below like # regens timed out waiting to be sent 
to PG is to have infrastructure elements like Queue instrument the current # items in the queue. 



• CM 




o 


# normal placements 


o 


# regen placements 


o 


.# instance expirations caused by clear cache 


o 


# instance expirations caused by template update 


o 


# instance expirations caused by record update 


o 


# instance expirations caused by periodic expiration 


o 


# failed regenerations 


o 


# regenerations that time out waiting to be sent to PG 


o 


# regenerations that time out waiting on PG response 


o 


list of active metadata clients 


Q 


list of disconnected metadata clients (perhaps)' 


• PG 




o 


time to generate . 


o 


# cache hits 


• WSplugin 


o 


# cache hits 
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13 Open issues 



13.1 Yicket 59722(CUST: Page regeneration has probably never worked correcdy on 
"Windows NT/2000) 

Thijs is about getting cached file last access time via Wjn32 HndHretRle vs, 
GetFileMomiationByHaiidle. The fonner can return a stale value. What does Java use? Try to 
reproduce the problem with COD. 

13.2 Vicket 8233(CUST: CMD should delete directones when template is 
expired/deleted) 

When metadata-msg.template-entry.template-event is delete, make sure to delete the 
cacheRoot/path AtPg subdirectory after deleting all the instances. 

13.3 Clearing WebseryerLookup table whenWS path to PG path mapping changes 
Do we add a new API command? Probably not since it should be extremely rare. • 

poes CM listen for an HTTP msg sent to placement port by some standalone utility? From email fix>m 
Isaac: • ^" . 

It may be better if we provide an option to the metadata tool to clear this table or perhaps clear .. 
for Sjpecific web servdr path patterns. Then the question becomes r how does this get refQled? I ' 
■ ' guess it should happen.ais a result of norma] placement because the web server will not find the ' 
cached page. A subsequent placement will be for an existing cached page - but, the CM should - 
always try to update the mapping table even if the placement can be ignored... 

Rather than learning the WS-PG path mapping, should we just add an XML element in tiie CDF or PDF 
to specify ws-path-pattem. to pg-path-pattem mapping? 

13 A Exploded deployment and CDF external to .warfiU deployment order problem. 

If a developer makes changes to the CDF and a template to add sensitivity parameters, and if the 
template changes are deployed before the CDF changes, there exists a potentially agnificant timing 
window where requests containing one of the new parameters will produce output dependent on the new 
paiameter, but Ihe instance will be cached without the new parameter. So subsequent requests without 
the new parameter will return results that are only produced when the new parameter is present. 

The solution is to malce sure the CDF changes are deployed before or with the templ$tte changes. 

If a developer makes changes to tine CDF and a template to remove sensitivity parameters, and if the 
CDF changes are deployed before the template changes, there exists a potentially significant timing 
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window where requests containing one of the obsolete parameters will produce ou^ut dependent on the 
obsolete parameter, but the instance will be cached without the obsolete paramet^. Sp subsequent 
requests without the obsolete parametw will return results that are only producejd when the obsolete 
parameter is present 

The solution is to make sure the template changes are deployed before or with the CDF changes. 

If a developer adds sensitivity parameters to 1 template and removes parameters from another template 
and both templates belong to the same webapp (and therefore use flje same CDF), then the solutions for 
the 2 previous bullets are incompatible. ' 

The solution for all 3 cases is to make sure the CDF ch^oiges are deployed with the template changes. 
This requires vcm support for grouping changes to a subset of content items within a deployment group, 
into a single deployment. This functionality has use beyond COD, especially for exploded deployment 
It could wprk like Perforce^ changelist functionality. 

13.5 Tod mdhy cache files in a directory 

MigueFs testing on this issue may not be. complete. Sb far, the results suggest that our^cunrent scheme 
. of just 1 directory per template may be fine. Howeyer, we will continue testing to see if it becomes a 
problem. 

* . " • 

13.6 Configuration bootstrappirig 

What is the sequence for configuring COD components? We need to figure out the basic scenarios: • 



13.7 Configuration teardown 

What is the sequence for tearing down COD components? We need to figure but the basic scenarios. 

13.8 Failover 

How will we do failover.? It could be as simple as configuring a backup CM and sleepycat that can be 
started by 3^ party failover software. The backup CM and sleepycat thert take control of the repository 
and cache over from the primary. 
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14 Closed Issues 

14.1 Preventing a cached component in a cached template from being served directly 
from the WS cache 

Web-xml can map a URI pattern like /component/* to an eiror page. Templates with these paths can 
then only be accessed via include or component commands. If one of these components is cached and is 
also called by a cached template, then both the template output and the component output will b^ cached 
in the WS cache. If a browsw directly requests the component, our plugin should not serve the request 
from the cache, 

The solution is: 

• the templateis invoked via the component conmiand, a string like "component" will be added 
to the request data to be hashed into the filename. 

• When handling SSIs, the plugin will add "con^onent" to the request data to beLhashed. thus 
matching the filename in the previous step. 

• When handling direct requests, the plugin will *not* add "component" to the request data to be . 
hashed, thus missing the component in the WS cachfe. . \ • 

If a component is not prevented from direct access by web.xml mi mapping, and if it's requested both • • 
directly and as a component, then 2 instances will go in the' cache, 1 with "component" in the filename & 
the other without - * ; ' / J 

14.2 Template output differs based on whether invoked directly or as a coniponent 

If a template's behavior differs based on whether it's invoked directly or as a component, then the 
template must contain logic like this: 

if ( request.getReguestURK) == ^/nrymu, jsp*' ) 

,ou t. print in ( "one thing" 
else ' 

out.printlnC ^another" ); * 

Do the following things to make sure output is properly cached: 

• Use web.xml to map 2 URIs to the template - 1 for direct access and another for component 
access. 

• Add an entry for each URI to the CDF. 

• Use the component URI in component conamand (as opposed to the unmapped URI). 
• • Use the compunejit URI in compaiisons with HltpSei'vletRequesLgelRec|uestURI. 
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TTiis also allows the direct and component instances to have different caching parameters. . . 

143 Vicket 57778(page regeneration doesnt do what it says on the tin..) 

iPlanet & IIS apparently cache pages in memory. When they serve from memory, our disk file's last 
access time doesn't get updated. For now, the solution is to explain the situation so the customer can 
work around the problem. See section 10.4 regenerate-last-accessed-interval for my attempt to do so, 

14.4 Record state guid 

The following scenario shows the record state guid being used to avoid caching a stale file. A page 
generation using old content completes (event 1) just before new content arrives (event 2). But CM 
processes the record update (events 3-4) before the placement with old content (event 6). CM then 
ignores the old placement (event 6) since its record state guid is less than the metadata record state guid. 
A page generation that occurs afl^ the record update (event 5) yields a placement (event 7) that is later 
accepted (events 8-9), 

r * . • . • 

d^loyment #n, rcdl 




14.5 Clear cache ID 

The following scenario shows how PG generates a unique clear cache ID to handle a race condition 
between placement and clear cache. 
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A ten^late executes on PGl and grabs old data. The data is updated, then clear cache is executed on 
PG2. PGl broadcasts the placement built from old data PG2 broadcasts the clear cache. CMl receives 
aud processes the placement before the clear cache. CM2 receives and processes the clear cache before 
the placement CAC rejects the placement \yilhxcId=oc since it doesn't ipateh md^ 

14.6 Instance version 

The next 2 scenarios show cases where a version mismatch .between the cache file and the instance 
metadata record occurs unless multiple instance versions arc allowed In each case, there is a window in 
which WS or PG can serve a cache file generated froni content version n with instance metadata 
produced by content version n-1 . ^ The sensitivity parameters in the instance metadata will be the same 
for both versions (oflierwise it would be a different instance), but non-sensitivity instance data like 
content tracking IDs and RMS events can be different for same instance when generat(E5d with different 
versions of contCTL . . 

The solution is to temporarily keep both the old and new version of the instance metadata and cache file. 
The instance version will be included in the cache filename not by including it in the parameter hash, but • 
by appending it modulo 10 to the parameter hash (c,g., /cachcRoot/pathAtPg/paramHash_v.shtml where 



57 



Vignette Corporation Confidential 



V = version % 10). A WS or PG metadata query for an instance can return multiple rows (probably 2 at 
most). The latest version will be used when serving the request. The version is a counter incremented 
during placement. The pld version will be deleted long enough after writing the new instance metadata 
to be reasonably- confident the instance metadata has reached all in memory database replicas (the 
default tuneout is several ordors of magnitude larger than the nonnal propagation delay). The timeout is 
controlled by the dJBANTJPJDELAY config variable described on pa^ 45. 

14.6.1 Regen placement overwrites file 

The mismatch occurs in a window that starts when Ihe cache file is written with data from content 
deployment n (event 7) and ends after the instance metadata is written *and* propagated to the WS and 
PG (event 8+ replica propagation). • 



deployment #n, rcdl ' 




md 

n-1 
2 n 



izist md 



n-1 
3 xi'X 
8 n' 



Valid 
Retgen 
Valid 




4 regen(nj 



14»6.2 Nonnal placement overwrites file - 

The mismatch occurs in a window that starts when the cache file is written with data from content 
deployment n (event 1 2) and ends after the instance metadata is .written *and* propagated to the WS and 
PG (event 13 + replica propagation). 

Thp window is only open when regeneration occurs, because WS and PG serve from the cache when an 
instance is in the Regen state. Without regen, processing of the record update (event 9) would have set 
the instance state to Invalid. Invalid would probably propagate to WS and PG well before the new 
version of the cache file is written (event 1^), so there should not.be a problem in the no regen case. 
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deplQyment #n, xcdl 
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The diagram does not show thes itegeneration request corresponding to the instance state transition to 
Regen (event 9); That regen request is stillin progress when the 2hdhonnaI placeinent makes a 
preemptive strike to transition the instance state from Regen to Valid (event 13). When the regen 
placement arrives later, it's ignored, since a Valid instance already exists. 

14.7 Metadata record locking 

The 2 following scenarios show cases where concunency within CM requires in-memoiy DB record 
lockinjg to ensure correct behavior. 



14.7.1 Record update after CM starts writing cache file 

The following scenaiio shows how in-memory database locking handles a xacc condition whci'c a x"CCord 
update tiies to update the RecoidStaLe at the same time an old placement Uies to update the instaiice • 
metadata table. In tliis case, the RecordState update occurs first (event 6): The instance metadata update 
(event 7) detects the RecordState update and sets the instance stale to Regen (and sends a regen request) 
• ralhei' than taking the normal action of siiaiply setting the state to V^d. 
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rs 9 Re'cordState table 

The locking is controlled by the JDBC Cqnnectiqii isolation leveL. We probably want SERIAIJZABLE. 
Froni igie TiinesTen d^^ '.' ; - • * , 

When running at SERIALIZABLE transaction isolation leveU TimesTen holds aU locks for the duration of tbt 
transactiODy so: 

• Any transaction updating a row blocks out repdm and writers until the transaction conunits. 
A Any transaction reading a>ow blocks out whtei^ until the tran'saction CO 
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. In this scenario, the following transactions compete, and record update wins: 
record update (event 6): 

Records tat e rcdState = mdCon . getRecordState (rcdUpdt . getStageld, rcdut>dt . getCdsId) 
rcdState . setGuid (recdrdUpdateWEsg. getGuid) 

rcdState. write ^ ■ , . 

XQdCon. commit 

write instance metadata (event 7): 

RecordState rcdState « mdCon .getRecordState {placement .getStageld, 

placement . getCdsId) 
if rcdState . getGuid == placement * getRecordStateGuid 

instMetadata. state = Valid 
else 

if instance should be regenerated^ 
instMetadata*. state Regen 

else' 

instMetadata. state = Invalid 
. . . set other instance metadata fields - • • ' - 

ins tMetadata . newRecord 

mdCon. commit ■* ' 

if instMetadata, state == Regen, 

send regen to WS . i 

else if instMetadata. state Invalid, 

delete cache '.file . .. . " 

Since the update occurred V\ the instance metadata write sees the record state ^^smatch, and transitions 
to either Regen or Invalid. If the instance metadata write had occurred 1*^, the instance would have bfeen . 
written in the Valid state, then the subsequent record update would have either invalidated it ot started 
its regeneration. . ... . ♦ . 

14.7*2 Concurrent placements 

The previous scenario showed that the metadata DB transaction that writes instance metadata must 
verify the template metadata record state guid still matches after the cache file is written. The same is 
true of the other synchronization naechanisms (templateHash, metadataHash, and clearCacheld). The 
following scenario shows that a similar thing can happen with the version, but the solution is different 
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CM processes placements x and y concurrently on 2 different threads. Both threads read the instance 
metadata and see that there is no record (event 3), so both dedde to write version 0 of the cache file 
(event 4). 

14.7.2.1 Problem 1 r concurrent open and write to the same file by different threads In the same process 
Here are 2 basic options for solving the concurrent file abcess problem: ^ . 

1. Use file locking to serialize file access 

. 2.. Use some other mutual exclusion mechanism to serialize file access 

#1 seems simplest and cleanest. It would be be^t if this could be done with Java file access classes, but 
my initial experiments with java.io.FileQutputStream don't show the locking behavior or level of 
control we need to serialize the accesses... However, java.nio.channeIs.FileChannel has lock methods .. 
that may do the. trick. I haven't experimented.with them yet If we ,c£|n't count on jaya 14, or if the 1.4 
java.nio classes still don't cut the mustard, I guess we*ll have to^ • • ^ ' • ■ 

Initially, I planned to have the 2^^ thread that tries to open (he file bldck tiU'the 1^ thread closed the file, 
then th6 2 thread's open would succeed. It would actually be better if the 2^ thread's open returned 
immediately or threw a lock exception, so the 2"^ thread could give up on the redundant placement 
immediately. That avoids problem 2 below. 

If imtnediate lock detection is not possible, the threads not getting the lock should block until 
closeOu^utStieam is called. Then see ProblCTi 2 below. 

14.7.2.2 iProblem 2 -concurrent insert of same row into instance metad^ . 

If the cpncuircnt file access problem is solved so that the 2"^ thread simply overwrites the file written by 
. the r"* thiead. Tlien both tlueads ti y Lo u ealc a new instance metadata row. That's tlie ucaL problem - . 
the 2"^ thread's insert will fail. Here are 2 basic options for solving the problem: 

.1. If the failure iR "duplicate record/' just ignore it. 
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c . c 

2. Check the version in the trsansaction doing the new record, arid roll back if it changed since it was 
. . read to calculate the cache file version. 

I'm going with the ''ignore the enx>f' solution, because it doesn't require an extra query for the nonnal 
case when this race condition doesn't occur. 

14.7.23 Adding a new state 

Another option for solving the concurrent placement problem is to add a Placing istate to the instance 
state diagram. Before writing the cache file, the placementMsgHandler writes the instance metadata 
With a state of Placing in the same transaction that queries for the max Version #. Another thread trying 
to place the same instance sees the new instance in the Placing state and realizes itis a redundant 
placement and abandons it 

The •'file locking*' solution is simpler, and it's definitely better as long as the locked file can be detected, 
immediately. Immediately abandoning the placement fiees the 2"*.thread for other work. 

If the lock can't be immediately detected, then Td probably still go with the fUe locking solution for 
simplicity's sake, but that introduces the performance^penalty of duplicate file Writes, and the 2"** thread 
also loses the time spent blocked waiting for the 1*^ thread's write to complete. If tiiose things cause 
performance problems in the real world, we could always switch to the "add a new state" solution. 
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U8 File locking 

14.8.1 V6 YS COD file handUing 

• V6 - . . 

o Bxjpite without regen: rename curLhtml -> curl-html.bak 

■ IfT- uses a quite complicated method MTmv: 
tries MoveFileExW; returns if ok 

open src fi]e & get a file mapping; share delete, read, & write 
open dst file w/truncate, share delete, read, & write 
if dst file not found, open w/ereate new, share delete, read & write 
for each chunk of src, MapViewOfFile, WriteFile chunk to dst 
close src file (& mapping 

open existing src file, delete on close, share delete, read, & write 
close src 

■ Other -VU_OS::rename. 

• o Expire with regenfrename temp.html F> curl .html 

■ Calls callPageDaemon in clients/common/generatePagelmpLcpp w/flag that WS 
writes cached pages 

• yU_OS jenameCtmpRle, cuylFile, removeSrc=IO 

o NT - CopyFileW(tmpFile, curlHle, faiIIfExists=F); 

.wunlink(tmpFiIe) • . ' • 

o Other -rename 

* VU_OS_unlink(curl.html.bak> 

» , o NT--_wunlink . . ' 

. o Other ^unlink . / 

• COD _ 

o . Normal placement: 

■ Multiple threads can concurrently try to open & write the same file. . Another 
process will not have the file open, because it's a new version. 
cmxreate/closeOutputStream will try to use java.nio file locking when creating 
the file s^t. 2"*^ thread is blocked until stream is closed. 2^^ thread's open then 
succeeds & 2^ create truncates the file & rewrites it. 

o Regen placement: 

" Create new version of file. Don't worry about multiple threads creating at once, 
since there will only be J regen placement 
o Expire without regen: . 

■ Listance deleter thread deletes file. javiLio.Filc.delctc fails if dst is open by 
another jvm, but it should still do since deleter doesn^t delete instances till enough 
time has passed to be assured no one is still using the instance. 

Resolution; i nstance version, instance state, and the instance cleanup thread collaborate to avoid the file 
locking problems encountered by V6. 
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14.8.2 Vicket 12365(nS file loddng interferes w/ PAD, CMD manipulating files) 

It sounded like this problem may have gone $way in an US update since Jam wasnit able to reproduce 
the problem with nS 4 or 5. ff this was just an ns bug that was fixed in IIS 4, then hopefully it wont be 
a problem for cop. 

Try to reproduce the problem with COD, If it is a problem, it wiU get interestmg since we'll have to use 
JNI for lower-level Win32 file hmdling- Here's an ongoing email thread on the issue: 

Original Message 

Prom: Rajkumar, Isaac 

Sent: Friday, January 18, 2002 3:12 PM 
To: Afshar, Jamshid; Caldwell, David 
Subject: RE: AFM and COD 

I wa3 just commenting on the reproducibility - we need to keep track of this for 
COD (like David hafe done). Thanks. 

-Original Message - 

Prom: Raj kianar, Isaac - ^ ' ^ . 

Sent: Friday, January 18, 2002 3:09 PM ' 
To: Afshar, Jamshid; Caldwell, David 
Subject: RE: APM and COD 

This fix was in 5.0.x and lost' in 5.5. - that is over 1-1/2 years of 5.5. and. this • 
is the first time. I am hearing about this. So, it looks like. this is a pretty rare 
occTurrence that is not easily reproducible - 

— ' — Original Message • - . 

Prom: Afshar, Jamshid ' . 

Sent: Friday, January 18, 2002 3:04 PM , 
To: Caldwell, David; Rajkumar, Isaiac 
Subject: RE: AFM and COD 

Although I wasn't able to reproduce, a customer is seeing what appears to be the 
same problem (we have Quickf ix 59939 for Banco De Calicia) . Also, 1 just heard from 
Mark that QA is able to reproduce it. 1^11 let you know what happens... 

Resolution; 

Instance version, instance state, and the instance cleanup thread obviate this problem. 
14.9 WebServerljOokiip Cohcurremy 
14 Concurrent placement-msgs . 

If 2 PGs concurrently generate a dynamic or cacheable page, 2 threads can end up trying to add the same 
. WebServerLookup row at the same time. The solution is to check for existence of the row before 
inserting it, where both the check and the insert are in the same transaction. 
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14.9.2 Concurrent template delete metadata-msg and placement-i^ 

If a PG generates a cached page at the same time a template delete metadata-msg occurs, 2 threads can 
end up trying to insert and remove WebSCTverLookup rows for the same template at the. same time. Hie 
solution is for PlacementMsgHandler to check for existence of the template inetadata in the same 
transaction that inserts the WS lookup row. If the template metadata doesn't exist, the lookup insert is 
aborted Furthermore, MetadataMsgHandler deletes the WS lookup data for the template in the same 
transaction that deletes the template metadata. 

14.93 Concurrent template add nietadata-msg and placement-msg 

If a PG generates a dynamic page at the same time a template add met^data-msg occurs, 2 threads can 
end up trying to insert and remove WebServerLookup rows for the same template at the same time* The 
solution is for PlacementMsgflandler to check for existence of the template metadata in the same * 
transaction that inserts the WS lookup row. If the template metadata exists, the lookup insert is aborted.' 
Furthermore^ MetadataMsgHandler deletes the WS lookup data for the template in the same transaction 
that adds the template metadata. 

14 JO Redundant metadata-msgs 

Multiple AMs send metadata-msgs to CM; 1 AM goes down for a while; template changes are deployed; 
CM clears the cache appropriately; lots of new instances are cached; AM comes back up & starts • 
sending the "old" template update messages to CM!, . • . . . . 

ilere is a solution that prevents CM from ■repeating;the clear cache operations unnecessarily (tMd = 
•'template metadata, m(ti4sg = metadata message): -.'.v - ' 

1. Add objectid and modCount colurims to template metadata . . 

2. AM includes objectid and modCount in template add/update/delete metadata-msgs to CM (In 
exploded deployment these come from the template itself. For templates in a war file, they are 
copies of the war file's objectid and modCount). . 

3. CM.MetadataMsgHandler.addTemplate: . ' 

a. if tMd exists, return 

b. add the template md, etc 

4. CM^deleteTemplate: . 

a. if iMd does not exist, return 

b. if tMd.objectId !=mdMsg.objectId, return 

c. delete the template md, etc 
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5. 



CM.updateTemplate: 



a. 



if tMd does not exist, return 



b. 



if tMdobjectId mdMsg,objectId, return 



c. if tMd.modCount >= mdMsg.modCount, return . ' . " . ■ 

d. if iMdtempIateHash = indMsg.temp]ateHash/reti^ 

e. update template md, etc 

6. CM.updateTemplateMetadata - same as updateTemplate except step^4 compares metadataHash 
instead of tempIateHash 

7. CM stores mdMsg objectid and modCount in tMd when it adds or updates the template or 
template md ' 

Why is objectid required? Object id becomes important when a template in exploded deployment^ or an 
entire Avar file otherwise, is created, updated several times, deleted, then re-added If an AM goes down 
before the deletion, when it comes back up and sends CM tiie delete with a high modCount, if there's no 
objectid, it appears to be a new message since the modCount saved in the metadata went back to 1 when 
the teinplate or war was re-added. , . 

Why do updateTemplate and updateTemplateMfetadata check the hash if them Kerens ./ 

the scenario that uses the hash: - 

app.war created with objectId=m,modCount=l, It contains: . 

ajsp with hash=a ....... 

b jsp vrtth hash=b 
This results in the following metadata: 

ajsp with objectId=m, modCount=l, hash=a • 
. b.jsp with objectId=m, modCount;=l, hash=b 
app.war updated with objectid^, modCount=2. It contains: ' 

ajsp with hash=c 

b jsp with hash=b 
This results in the following metadata: • 

ajsp with objectid=m, modCount==2, hash=c 
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b jsp with objectId=m, modCounfc=l, hash=b 
Now suppose a new AM is added - . 

app.war is created with objectId=m, modGomit?=2. It contains: 

a jsp with hash=c 

b.jsp with hash=b ' 
AM sends a metadataMsg containing: 

ajsp with objectId=m, modCount;=2, hash=c 

b jsp with objectH=ni, modCount=2, hash=b - . 
. If CM just looked at objectid and modCount, it would unnecessarily clear all of b jsp's instances. 
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51'BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR Sn)ES 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

I^LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



